What does this PR do?

Stack: this PR → #50496 → #50511 → #50514 → #50499 → #50521 → #50522

Adds transformers/distributed/context_parallel.py, the whole context parallel (CP) module. Nothing is wired yet: no DistributedConfig field, no plan on the configs, no cp mesh axis. #50496 brings those.

What this PR brings

• ContextParallelLayer, the base of the styles below, with the install_forward contract of the TP and EP styles: it replaces module.forward by one that runs transform_inputs_pre_forward first (with functools.wraps, so inspect.signature(model.forward) stays what Trainer expects).
• "cp_split_sequence" (SplitSequenceParallel), for decoder layers: they run on this rank's contiguous slice of the sequence. The first layer of the plan cuts hidden_states, and every layer cuts the position_embeddings. The attention mask and position_ids stay those the model built for the whole sequence.
• ContextParallelAttention, the base of the attention styles: install_forward checks the module, gives it attention as its cp_attention, and wraps the model's entry in ALL_ATTENTION_FUNCTIONS so that the attention call of that module runs through it. The algorithm is a plan style, like a TP style: "cp_ulysses" (UlyssesAttention) all-to-alls the query, key and value to the whole sequence with num_heads / cp_size heads, runs the model's own attention function with the model's own mask, and all-to-alls the output back. Sliding windows, chunked attention, padding and packing therefore need no CP mask code. Another algorithm is a subclass registered in ALL_PARALLEL_STYLES and named in the plan; context_parallel_styles() lists them for the plan validation.
• ModelContextParallel, the style of the model itself, outside the plan since no plan key names the root module: its forward refuses a cache, and pads every sequence at its end to a multiple of cp_size before the model builds its masks (input_ids get pad_token_id or 0, inputs_embeds and attention_mask zeros, labels and shift_labels -100, position_ids continue the last document). The logits are this rank's slice of the padded sequence.
• context_parallel_loss, set as model.loss_function: every rank reports the loss of the whole sequences, computed from its slice of the logits and its slice of the shifted labels, with a gradient of this rank's times cp_size, so that FSDP2's average over the cp ranks is the gradient of the full-sequence loss.
• apply_context_parallelism(model, cp_mesh, cp_plan): refuses what CP would silently get wrong (attention that does not go through the attention interface, layer types other than softmax attention, heads other than a causal LM), then installs ModelContextParallel and the loss on the model and the plan's styles on their modules.
• all_to_all and split_sequence, the two helpers the above use. The all-to-all is torch's all_to_all_single_autograd, which EP uses too.

What #50496 brings

DistributedConfig(cp_size, cp_plan), base_model_cp_plan on the configs and the mixin's plan resolution, the cp mesh axis with FSDP2 over fsdp × cp, the models that opt out, the Trainer refusal, and the end-to-end tests through from_pretrained(distributed_config=...). It adds no code to context_parallel.py.

Tests

tests/context_parallel/test_context_parallel.py, on 2 CPU ranks (gloo):

• the all-to-all and its backward;
• Ulysses against sdpa on the full sequence with the full mask (causal, packed, sliding window, GQA), forward and backward;
• the keyword arguments passing through unchanged (sinks are cut to this rank's heads);
• the split style, the routing of the attention call through an attention style, the Ulysses checks and the model checks, the registered styles;
• the loss against the full-sequence loss: value, gradient, a caller's num_items_in_batch, a slice whose shifted labels are all ignored;
• a tiny Llama with the plan installed by hand, against the same weights on the full sequences, with 8 and with 7 tokens (padded to 8): this rank's logits at the real positions, the loss and every gradient; the cache is refused, model.forward keeps its signature;
• what the padding does to each input, single process.